Can We Derive General World Knowledge from Texts?

As one attack on the “knowledge acquisition bottleneck”, we are attempting to exploit a largely untapped source of general knowledge in texts, lying at a level beneath the explicit assertional content. This knowledge consists of relationships implied to be possible in the world, or, under certain conditions, implied to be normal or commonplace in the world. The goal of the work reported is to derive such general world knowledge (initially, from Penn Treebank corpora) in two stages: first, we derive general “possibilistic” propositions from noun phrases and clauses; then we try to derive stronger generalizations, based on the nature and statistical distribution of the possibilistic claims obtained in the first phase. Here we report preliminary results of the first phase, which indicate the feasibility of our project, and its likely limitations.